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Letter from the State Superintendent 


Dear Maryland Educators, 


The work to support student learning in Maryland is crucial. As a community, Maryland is committed to supporting a world-class 
educational system in which teaching and learning take place every day in safe, supportive, and respectful school environments, and where 
students, educators, and families are valued and have the opportunity to succeed. Positive climates and successful school discipline 
policies and practices administered fairly, equitably, and consistently support positive student outcomes. Therefore, school should not use 
discipline practices that disproportionately impact specific students, including but not limited to students of color and students with 
disabilities. Where such disparities exist, school systems must make continuous efforts to understand the causes of and reduce such 
disproportionality. 


Over the past years, the Maryland State Board of Education (MSBE) has examined the negative effects of suspensions and expulsions. In 
a report entitled A Safe School, Successful Students, and A Fair and Equitable Disciplinary Process Go Hand in Hand, the MSBE 
determined that a significant percent of out-of-school suspensions were for non-violent offenses and the disproportionate impact of school 
discipline seemed to fall upon students of color and students with disabilities. Maryland has substantially reduced the use of suspensions 
and has decreased the use of suspensions for non-violent offenses. However, the disparate impact for students of color and students with 
disabilities remain. 


Over the past 20 years, the racial composition of Maryland school has changed. Therefore, it is imperative that school systems build the 
necessary infrastructure to support all students. The Maryland State Department of Education (MSDE) is working to provide support in 
the area of addressing issues related to disproportionality. This workbook is designed to support school and local systems in examining 
policies, procedures, and practices for serving students from diverse backgrounds and students with different abilities. 


Safe and positive school environments are built on trusting relationships between students and school staff. When students experience a 
system of school discipline that is administered fairly, equitably, and consistently, they will more often accept responsibility for their 
actions and the consequences resulting from their behavior. Together, we can continue to make progress toward ensuring Maryland 
students are educated in a world-class educational system where teaching and learning take place every day. 


Karen B. Salmon, Ph.D. 
State Superintendent of Schools 


Introduction 


The MSBE has challenged itself and schools throughout the State to create a world class education system that prepares all of Maryland’s 
students for college and career success in the 21st Century. To realize this challenge, we must create schools that are both welcoming and 
academically rigorous, in which are provided the educational and social tools they need to succeed. 


In July 2012, the MSBE issued a groundbreaking report on school discipline practices declaring that school discipline and academic 
success are equal partners in education reform. The report noted that school discipline practices, particularly out-of-school suspensions 
and expulsions, are linked to academic achievement for the simple reason that in order for students to have an opportunity to receive a 
world class education, they first and foremost need to be in school. 


When a student is excluded from school, the exclusion impacts not just the student, but also the family and the school. For the student, 
suspensions are a leading factor in a student’s decision to drop out of school. For the family that has young children, the risks associated 
with an unsupervised child must be balanced against the financial cost of missing work to stay home with that child. For the school, 
exclusionary discipline is often associated with less satisfactory ratings of school climate and negative achievement outcomes. When 
schools rely too heavily on exclusionary discipline, the impacts are both negative and cumulative.' 


To spark reform in the area of equitable discipline, the MSBE adopted a regulation in January 2014, which directed the MSDE to develop 
a method to analyze school discipline data to determine whether school discipline practices are having a disproportionate impact on 
students of color and students with disabilities (COMAR 13A.08.01.21). 


COMAR 13A.08.01.21 


impact on minority students. 
B. The Department may use the discrepancy model to assess the impact of discipline on special education students. 


The Department shall develop a method to analyze local school system discipline data to determine whether there is a disproportionate 


If the Department identifies a school’s discipline process as having a disproportionate impact on minority students or a discrepant impact on 


special education students, the local school system shall prepare and present to the State Board a plan to reduce the impact within one year 


and eliminate it within three years. 
D. The local school system will report its progress annually to the State Board. 


Maryland State Board of Education, School Discipline and Academic Success: Related Parts of Maryland’s Education Reform, July 2012, available at: 
http://www.marylandpublicschools.org/stateboard/Documents/StudentDiscipline/SchoolDisciplineandAcademicSuccessReport0712.pdf 


Purpose 


Disproportionate rates of suspensions and expulsions among racial/ethnic groups are a local, state, and national concern. Maryland has 
made significant progress in decreasing the use of exclusionary discipline, but there is substantial work left to do to move toward equity. 
Maryland has cut the use of exclusionary discipline practices by half in the last decade, from 8.7 to 4.3 percent since the 2005-2006 school 
year. However, when Maryland discipline data is disaggregated by student group, it becomes apparent that the experience of individual 
student groups varies widely. 


According to 2015-2016 data, 8.1 percent of African-American students and 9.9 percent of students with disabilities in Maryland received 
an out-of-school suspension or expulsion, compared to just 2.3 percent of white students and 3.6 percent of students without disabilities. It 
was review of data such as this that sparked reform in the area of equitable discipline and caused the MSBE to direct the MSDE to develop 
a method to analyze local school system discipline data. Data indicators can help school climate or leadership teams identify racial/ethnic 
disproportionality in school discipline practices. 


The purpose of this guide is to help educators understand the MSDE’s method for analyzing out-of-school suspension and expulsion data 
to determine whether school discipline practices have a disproportionate impact on students of color and students with disabilities. This 
guide outlines a process for examining disproportionality within schools, as well as root cause analysis to help systems and schools 
identify areas of need on which to focus to reduce and eliminate disproportionality in schools. The guide is a part of a statewide effort to 
reform school discipline and ensures that policies and practices related to student conduct foster appropriate behavior create a positive 
school climate, and safe environments. In addition, this guide is a framework for Maryland schools and systems to use in analyzing 
discipline-related policies and practices that may lead to disproportionality. 


The Maryland Model 


The Maryland model to determine disproportionality provides a comprehensive way to analyze out-of-school suspension and expulsion 
(removal) data using two complementary measures: risk ratio and State comparison. The term “removal” refers to an out-of-school 
suspension or expulsion. In-school suspensions and other forms of school discipline are not included in the Maryland model. The data to 
be used in the Maryland model will be unduplicated. The number of students who received one or more out-of-school suspensions or 
expulsions will be counted, not the number of out-of-school suspensions or expulsions issued. The Maryland model focuses on the 
unduplicated number of students because for some student groups, the risk of removal is higher the very first time they face school 
discipline. Even if a student has no history of misbehavior, if disproportionality exists, then removal is a more likely response to that 
student’s behavior simply by his or her color or disability. 


The Maryland model analyzes removal data for all students using the risk ratio and State comparison measures. Utilizing the same model 
for general education and special education students alike is an important consideration for equity. By analyzing removal data at the 
individual school level, the MSDE continues to support the transformation of school climate by viewing it, in part, through the added lens 
of disproportionality. Analyzing data through this lens is a critical first step as schools begins to develop specific interventions to address 
root causes of disproportionality and school systems institute data-informed plans to reduce, and ultimately eliminate, disproportionality in 
school discipline for all students. 


Getting Started 


To make sustainable change that will reduce and ultimately lead to the elimination of disproportionality, local education agencies (LEAs) 
and schools must create and implement a process for systemic change through the use of well-established and highly functional 
disproportionality teams. The local school system-level and school level teams must have the knowledge and technical expertise to 
develop a deep understanding of disproportionality, root causes, data, and strategies to address problems. Deming (1986) conveys it is 
essential to plan and evaluate through a continuous improvement lens by examining pertinent data points, asking questions, and attuning 
interventions towards mastery. To aid in the facilitation of this process, educators will use the four-step Plan-Do-Study-Act (PDSA) as the 
framework to guide this work. 


Plan Do Study Act Cycle 


The Plan Do Study Act Cycle is a cyclical four stage improvement process. The cycle begins with the Plan stage which includes 
identifying root causes, developing goals, creating an intervention plan, and choosing indicators to track progress. The Do stage includes 
implementing the intervention plan and collecting indicator data to monitor fidelity of implementation and tracking progress. The Study 
stage consists of evaluating progress, reviewing what has been learned, and determining what adjustments, if any, is needed. The Act stage 
is used to adjust the intervention plan if needed. These four steps are repeated as part of a cycle of continuous improvement. 


During the 2017-2018 school year, the Plan Do Study Act Cycle will be completed in a unique format to align with the phased 
implementation of the Maryland model. At this point, the Plan stage includes analyzing removal data, identifying root causes, developing 
goals, investigating evidence-based interventions, and choosing indicators to track progress. The Do stage takes a deeper look at 
interventions in order to pair them with the identified root causes and select a data tracking process. The Study stage requires reviewing 
current practices and what has been learned. Finally, the Act stage includes making adjustments to any interventions, as needed, in 
anticipation of full implementation of the Maryland model in the 2018-2019 school year. At that point, schools will be identified and will 
develop their plans to reduce disproportionality within one year and eliminate it within three years. 


2017-2018 
PLAN: Step 1: Identify Root Causes STUDY: Step 3: Review Current Practices 


DO: Step 2: Develop and Identify Alternatives to Removal ACT: Step 4: Adjust Interventions 


The illustration below outlines the framework: 


Figure 1: Plan—Do—Study—Act: Using data to guide improvement decisions 


What are How will we What changes 

we trying to know that can we make . 

accomplish? a change is an that will result in 
improvement? improvement? 


Source: Authors’ illustration of the Plan—Do—Study—Act continuous improvement cycle, based on Deming 


A detailed explanation of each the steps along with recommendations will be included in the next sections. The process should begin with the LEA 
identifying a point of contact to be a conduit for information from the MSDE about removal data, professional development, and next steps. The 
LEA and schools should assemble Disproportionality Review Teams (DRT) to complete the 4 step process. 


The MSDE is using a three phase approach to provide technical assistance to the LEAs. The initial phase provides an overview of their 2016-2017 
data. Phase two in 2017-2018 allows for a deeper dive to determine the root cause analysis and phase three addresses full implementation of an 
action plan for reduction, and elimination in school year 2018-2019 and beyond. 


The illustration below indicates an overview of the three stages the MSDE will execute towards the identification of schools, reduction of in year 
one, and elimination of disproportionality in three years. 


e ‘January 2017 data dissemination 


° Information gathering 
° Professional Development 
° Stakeholder engagement 


Phase 1: Initial Data 
Review 


e — School year 2017-2018 Root Cause Analysis by school systems and school 
Phase 2: Root Cause level teams 
Analysis ° Professional development and technical assistance provided by the MSDE 


e Stakeholder and partner engagement through pre and post surveys 


Phase 3: Full 
Implementation 


° School year 2018-2019 Full implementation: 

° Identification of schools 

e Submission of strategic plan by local school system to reduce and 
eliminate disproportionality 


The MSDE delivered school level removal rates to each LEA to help identify trends and patterns in January of 2017. Professional development 
was also provided to LEAs on how to interpret their removal data and to familiarize staff with the risk ratio and state comparison measures. 


Given the variation in data that may exist between schools, the LEA points of contact should train the school level DRTs. The process of 
assembling teams at the individual school level may occur after the LEA disproportionality review team is assembled and functioning. One of the 
strengths of the Maryland model is that it analyzes removal data at the individual school level, where school discipline practices are applied to 
students. School staff must also work together in analyzing the data to identify root causes for disproportionality among students and engage in 
difficult conversations about why students of color and students with disabilities are more likely to receive exclusionary discipline. The 
membership of a school DRT should be diverse. Schools may choose whether to identify interventions that are effective in reaching one student 
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group of concern or focus on school-wide reforms. The school team is fundamental to this work, because it is perhaps most capable of working to 
address identified root causes of disproportionality related to staff actions, perceptions, and biases. 
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Form LEA and School Level Disproportionality Review Teams (DRT) 


Katzenbach and Douglas (2015) states that, “a team is a small number of people with complementary skills who are committed to a common 
purpose, performance goals, and approach for which they hold themselves mutually accountable.” An efficient team is essential to produce high- 
quality outcomes. All members of the school community have a role in building schools where teaching and learning take place every day in safe, 
supportive, and respectful school environments, and where students, educators, and families are valued and have the opportunity to succeed. Our 
schools are the safest and most successful when everyone collaborates, respects each other’s roles, invests in positive, proactive behavior support 
practices and develop clear, objective school discipline practices. Meaningful parent, student, and community involvement in the creation and 
application of school and school policy is essential for building an effective school with positive and inclusive learning environments. 


Implementation of the Maryland model for disproportionality requires two types of disproportionality review teams. Each LEA will be 
responsible for assembling a DRT at the LEA level as well as the individual school level. The LEA DRT will provide strategic direction for 
school DRTs by analyzing removal data to identify common areas of concern and leveraging resources to support individual schools. Each school 
DRT will take a deepest dive into its own data to identify root causes and begin conversations about how to change culture, climate, and school 
discipline practices as applied to its student population. The implementation of alternatives to removal at the school level is critical to reducing, 
and ultimately eliminating disproportionality. The table below outlines possible members to serve on the LEA and school DRTs. 


All DRTs should develop communication protocols, describe the roles and responsibilities of each member of the team, and create regular meeting 
dates and times. It is critical that the team also uses data to (a) help identify valued outcomes based on need, (b) monitor fidelity of intervention 
implementation, and (c) monitor the impact on youth and adult outcomes (data). Highly effective teams take the time to develop the group. The 
local disproportionality review teams will receive support from the MSDE (e.g. ongoing professional learning, coaching, and performance 
feedback, etc.) to fully implement and monitor practices that will ultimately address issues of equity. When assembling members of the team, it is 
important to have a diverse DRT in terms of roles (e.g. student service director, special education director, local accountability coordinator, equity 
officer, etc.) and demographics (e.g. individuals of color, individuals with disabilities). To the extent possible, a DRT should reflect the diversity 
of the LEA that it serves. Diversity will help support substantive conversations regarding disproportionality including answering questions such as 
“why racial/ethnic minority populations or students are with disabilities being disproportionately affected policies, practices, and beliefs?” 
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Potential Local Education Agencies (LEA) Disproportionality Review Teams (DRT) members could include: 


LEA Level Members Task Timeline 
Superintendent (or After the review of the data, establish a clear e August 2017 
representative, i.e., Asst. and concise goal for stakeholders, provide 
Supt., Student Services sufficient human and fiscal resources for team 
Director, etc.) members to begin to plan for possible 

implementation of evidence based strategies in 
consultation with all parties. 
Representatives of Local Provide support to the superintendent’s goal to e August 2017 
Boards of Education and eliminate disproportionality through 
Union Leadership collaborative discussions and resources. 
School Level Leadership Identify team members with expertise in e August 2018 


of Identified Schools 
(including Principal, 
Assistant Principal, and 
Deans of Discipline) 


addressing the educational needs of individual 
students who exhibit behavioral concerns. 


Identify team members who have experience 
using and designing instructional techniques 
that address the individual needs of students. 


Facilitate an effective review of the data, 
current practices, and the purpose of the team. 


Monitor the effectiveness of the process, 
review of interventions and their 
implementation status. 
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LEA Level Members Task Timeline 
LEA Positive Behavior Coordinate the implementation of positive August 2017 
Interventions and Support behavior initiatives with fidelity. 

Point of Contact 
General Education and Review classroom disciplinary practices. August 2018 
Special Education 
Teachers (elementary, Implement positive behavioral initiatives. 
middle, and high school) 
Collect and review student data. 
Representatives from Identify individual students who may need August 2018 


teacher/intervention teams 
and special education 
review committees 


assistance. 


Implement interventions or consult with other 


staff members who may provide small group 
or classroom assistance. 


Parent(s) and/ or local 
parent groups 


Share knowledge about the child’s strengths 
and needs. 


Provide information about the family’s 
cultural background. 


Suggest strategies that have worked with the 
child. 


Implement strategies in the home. 


August 2018 
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LEA Level Members Task Timeline 
Local college or university Provide additional support, research, and e August 2018 
faculty technical. 

LEA or community groups Provide researched based best practices to e August 2017 


that provide support to the 
LEA and/or students who 
are disproportionately 
represented 


assist the team with resources for 
implementation. 
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Potential School Level Disproportionality Review Teams (DRT) members could include: 


School Level Members Task Timeline 
School Level Leadership Identify team members with expertise in August 2017 
of Identified Schools addressing the educational needs of individual 
(Principal, Assistant students who exhibit behavioral concerns. 

Principal, and Deans of 

Discipline) Identify team members who have experience August 2017 
using and designing instructional techniques 
that address the individual needs of students. August 2017 
Facilitate an effective review of the data, August 2018 
current practices, and the purpose the team. 
Monitor the effectiveness of the process 
review of interventions and their 
implementation status. 

General Education and Review classroom disciplinary practices. August 2017 


Special Education 
Teachers (elementary, 
middle and high school) 


Implement positive behavioral initiatives. 


Collect and review student data. 
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School Level Members 


Task 


Timeline 


Representatives from 
teacher/intervention 
teams and special 
education review 
committees 


Parent(s) and/ or local 
parent groups 


Identify individual students who may need 
assistance. 


Implement interventions or consult with other 


staff members who may provide small group 
or classroom assistance 


Share knowledge about the child’s strengths 
and needs. 


Provide information about the family’s 
cultural background. 


Suggest strategies that have worked with the 
child. 


Implement strategies in the home. 


August 2018 


August 2017 
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School Level Members 


Task 


Timeline 


Program Leadership: 
English as a Second 
Language, Title 
Programs, Special 
Education, School 
Counseling, School 
Psychology, Student 
Services, etc. 


Provide assistance to students as appropriate 
and in accordance with the school and LEA 
goals and objectives. 


August 2018 


School Counselors, Pupil 
Personnel Workers and 
Student Service teams 


Provide assistance to students as appropriate 
and in accordance with the school and LEA 
goals and objectives. 


August 2018 
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PLAN Step 1: Identify Root Causes 


Table 1. 


At this stage of the improvement process, it is essential to ask questions and come to consensus on the end goal. Disproportionality Review Teams 
should examine the data to gain additional knowledge about the issue. This can be done through a root cause analysis. The root cause analysis is a 
method LEAs and schools can use to identify underlying reasons for disproportionality. A root cause can be defined as "An objective, thorough 
and disciplined methodology employed to determine the most probable underlying causes of problems and undesired events within an 
organization with the aim of formulating and agreeing on corrective actions to at least mitigate if not eliminate those causes and to produce 
significant long term performance improvement." - Vorley & Tickle, (2002). The members of the DRTs should work together to review the data 
provided by the MSDE. The members of the LEA team will support the school level team to review the data provided by the MSDE upon request. 
A sample of the data chart is outlined in Table land Table 2 below: 


Elementary School Data 


State Comparison 
Student | Student} Student {AllOther/AllOther| AllOther | RiskRatio | Statewide a 


School | School | Student Measure 
Name | Number | Name | Type | Group 


Group | Group Students | Students} Students | Measure | Removal 
Removal | (A+B)+(X+Y) 
Rate (X +¥) 


(A+B) + (Statewide 
Removal Rate) 
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Table 2. 


Middle & High School Data 


School | LEA School | School | School | Student Risk Ratio 
Measure 
Year | Number Number | Name | Type | Group 
Removal 
Rate (X +¥) 


a a a a ae a ee ee ee ee 2 
a a a a a a ee ee ee se 
a a a a ee a a ee ee 2 
a a a ee ee - 
a ee ee ee ee eee eee 


What causes disproportionality? Research indicates the cited items in the illustration below serve significant impact disproportionality. 


Discipline 
Policies and Tracking 
Practices 


Teacher 


Interventions : 
and Referral epee 


Process oe Misconceptions 


Instruction, 
Curriculum and Disproportionality 
Assessment 


Cultural 
Dissonance 20 


Disproportionality teams should analyze data points to determine disproportionality trends among student groups. There are several possible 
reasons for disparity among groups. According to Voulgarides and Zwerger (n.d), common root causes for disproportionality are found in issues 
surrounding discipline policies and practices, intervention and referral processes, instruction and assessment, access to educational opportunities, 
family and community partnerships, teacher and administrator expectations, and cultural dissonance. Research has shown a direct correlation 
between an individual’s implicit biases and their actions in a real world setting. Implicit bias is defined as an unconscious judgment, based on 
stereotypes, beliefs, values, and ambiguous decisions. It is essential to use data to drive change in local education agencies and individual schools. 
Data-driven root cause identification aids DRT teams on the LEA and school level to take a deeper dive into identifying root causes of the 
problem, the impact on student groups, and the opportunity to develop and implement resolutions for equitable outcomes to discipline practices. 


The LEA and individual school teams should engage in the review of the data guided by the probing questions to identify what can be 
implemented structurally and what system strategies can be put into place to address issues of disproportionality. Probing questions should be 
designed to connect root causes for disproportionality with policies, practices, and beliefs. Below are questions that can guide DRT’s discussion 
on using data throughout the Plan-Do-Study-Act process. Each LEA and school DRT should ask these questions as applicable to their unique 
roles, so that individuals may focus on root causes that are within their locus of control. 


Are there systemic concerns contributing to the problem? Is the problem schoolwide or in isolated areas? 


e Are there specific protocols and practices that promote a positive school climate and equitable discipline clearly 
defined in policies and guidance documents? 

e Is the emphasis of discipline policies and protocols on prevention and keeping students in school? 

e Does the school implement schoolwide evidence-based practices and programs that support positive climates, student 
engagement (e.g., PBIS and Restorative Practices, etc.), and alternatives to removal to determine existing strengths? 

e Are there discipline policies and restorative practices to keep students in school? 

Do current discipline policies present barriers to potential solutions? 

e Does the LEA and school have the data necessary for identifying the root causes of problems, selecting evidence 
based solution, and tracking progress? 


What are the reasons or reported behaviors for which students receive suspensions? 


e@ What are the top three reasons for suspensions/expulsion overall within the school? 

Are there specific reasons for suspension of the top five suspension and expulsion codes? 

e How much more likely is it for racial/ethnic students of color to receive a suspension for disruptive or disrespectful 
behaviors, then other ethnic groups? 

e Are there external factors influencing the effectiveness or equity of school discipline practices? 
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What type of behaviors lead to exclusionary discipline? 


What typically occurs before and after the discipline incident? 
Does communication or cultural misunderstanding contribute to the problem? 
e Are the concerns schoolwide or in isolated settings and times? 


Do teacher classroom management skills or teacher- student relationships play a significant role? 


Do teachers provide positive feedback to each student? 

Do teachers possess high expectations for each student? 

Do teachers provide a rigorous, engaging environment for each student that exposes them to cultural differences? 
Do teachers establish ritual and routines on how conflicts can be resolved? 

Do teachers maintain cultural sensitivity when interacting with students? 

Are students subjected to negative, racial/ethnic group intentional/unintentional comments that are disrespectful? 


What practices are being implemented well? What practice needs improvement? 


Are processes that promote a positive school climate and equitable discipline practices adhered to across the LEA? 
Are schoolwide interventions available that could prevent discipline incidents, such as re-teaching rituals and routines, 
increasing supervision in specific locations, or changing schedules? 

Are there external or other factors influencing the effectiveness or equity of school discipline practices? 

Is the school or LEA executing professional development or targeted interventions to address cultural or other factors 
that may increase the likelihood of a discipline incident? 

Does the school collaborate with families and the community to maintain a welcoming educational climate that 
supports teaching and learning? 


How do you determine that LEA and school current intervention are being successful? 


How will interventions be measured to determine if school discipline intervention is completed with fidelity? 

What data points will inform the LEA and school if progress is being made toward the elimination racial/ethnic 
disproportionality? 

What data points will inform the LEA and school if there is a reduction in the use of exclusionary discipline, 
especially in specific problem areas in the school or LEA? 

What are the positive outcomes that the LEA and school hope to accomplish by reducing exclusionary discipline, such 
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as increasing classroom instructional delivery and time on task? 
e How do the LEA and school know if the LEA and school are achieving these outcomes? 


Once the LEA and school has identified root causes or specific conditions that lead to disproportionate discipline practices, it is time for the DRTs 
to gather information to inform the selection of evidenced- based or promising practices. After completing this process LEAs and schools should 
review and prioritize root causes. Disproportionality Review Teams should be culturally responsive, which requires a development of cultural 
competence, and habits of equity thinking in order to create effective practices. Conley (2005) conveys a practical approach for culturally 
responsive habits of thinking DRTs should consider. Culturally responsive habits of thinking are as follows: 


e Problem Identification: What is happening here? What are the cultural implications/results? Is there a 
disproportionate representation? 


e Reflection: How does the LEA or school level operate for various cultural groups? What has worked and 
what has not? What has happened here previously? 


e Interpretation: How is another cultural perspective of the problem likely to offer different guidelines for 
the system, behaviors, and individual choices? 


e Reasoning: How can the LEA and school validate or assess the new cultural approach? Checks for 
precision and accuracy throughout. 


For example: Code of Maryland Regulations (COMAR) 13A.08.01.11 (Disciplinary Actions) How does the LEA and/or school utilize 
discipline for infractions like “disrespect”? Utilize the above four steps to guide the discussion to make ongoing systemic practice and policy 
changes. 
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DO Step 2: Develop and Identify Alternatives to Removal 


Now that the LEA and school have identified the root causes for disproportionality, the LEA and school are ready to develop alternatives to 
removal. During the 2017-2018 school year, DRTs should choose evidence-based interventions that target the identified root causes. Consider the 
programs and practices cited in appendix A, which support positive school climate and student engagement for both general and special education 
students. If the root cause is explicit bias, then professional development for administrators, teachers, and other support staff should be conducted 
to address it specifically and focus not on student centered classroom management. 


Once interventions are selected, it is critical that be related professional development to occur ensure both buy-in and fidelity in implementation by 
administrators, teachers, and other support staff. Depending on the root cause identified, the focus of the training, and the audience for it, may 
differ. In addition, it will be critical to include all stakeholders, be parent and community engagement activities the LEA and school level. 
Disproportionality Review Teams should think about strategies on, how do you engage? Finally, based on the determined gaps in knowledge or 
skills, the DRT should identify technical assistance needs that may be provided by the MSDE. For example, does the LEA or schools identified 
require technical support to develop alternatives to suspension, student engagement, and classroom management for general education and special 
education students? 


One way to accomplish this goal will be to start with a small group of schools or students. Starting small can help the team gather information 
about the effectiveness of the intervention and make adjustments prior to going schoolwide or districtwide. The start small strategy can help a 
team determine appropriate professional development and support needed to implement the intervention with fidelity. 
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STUDY Step 3: Review Current Practices 


At this stage, DRTs at the LEA and school levels should use formative measures to check on implementation progress. The MSDE will provide 
the removal data for the LEAs and schools the fall of each school year. The DRTs at the LEA and school level should establish a process to collect 
data on specific indicators to effectively progress monitor to answer essential questions that drives the disproportionality improvement process: 

"Is what we're doing working, and how do we know?" Evaluation allows the DRT to look back over the process and ask the questions, "How did 
the intervention work, what was its impact on students, and what adjustments should be made?” Program evaluation depends on good monitoring. 
The Disproportionality Review Teams during 2017-2018 at the LEA and school level should not wait until the end of the year to see if the 
measures being implemented are working. The review current practices are a useful tool for checking on implementation and making adjustments 
for unanticipated events or barriers. Some guidelines for effective progress monitoring include the following: 


e Some formative techniques that are used to monitor implementation include: 


1. 


li. 


lil. 


iV. 


Vil. 


Review of student discipline data by student group, such as ethnicity, gender, general and special education, etc.; 
Review of student discipline referral data; 
Surveys of staff to determine how the continuum of responses to behavior is applied, 


Observations of students in both general education and special education settings to determine the most prevalent 
behaviors, the impact of the classroom setting on the behaviors, and responses to those behaviors; 


Checklists of student behavior to determine the frequency of behaviors in the classroom and other school environments; 
Student interviews to determine perceptions of adult responses to behaviors ; and 


External review teams to provide an objective view of school climate and its impact on student behavior. 


e LEA and school level teams are expected to schedule regular meetings to monitor progress. LEA and school teams should designate staff 
members, and representative stakeholders to review what’s happening and be cheerleaders for progress throughout the school year. These 
individuals should be members of the DRT. After review of the data, if progress does not appear to be occurring team members should 
discuss some possible reasons why and identify strategies for improvement. For example, are the identified programs and practices being 
implemented as planned? Do school staff need additional support in implementing programs and practices? Are data being collected 
consistently for all affected student groups? Were the appropriate root causes identified? 
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ACT Step 4: Adjust the Interventions 


Based on the analysis of the student data, quality review, disciplinary process, and interventions the DRT should examine the following: if the 
interventions are achieving desired results, if not make adjustments or discontinue them in their entirety and develop new ones. DRTs should take 
a deeper dive by looking at the trends of student groups such as: white students as opposed to African American students, etc. In this stage DRTs 
at the LEA and school level should review their data regularly to ensure that interventions are working. If the data reveals little or no progress, the 
team should investigate and make necessary changes. If the DRT was successful, then the team could consider expanding it or identifying a 
different problem of practice to address. In either of the above examples, this stage provides an opportunity for DRTs to begin a new cycle of 
planning based on observation. 


Based on the level of progress made during the 2017-2018 school year, the LEA will have some idea as to whether it is at risk of being identified 
in the 2018-2019 school year. As part of reviewing the success of interventions, it will be important to begin to outline additional technical 
assistance needs and what lessons have been learned that could be applied to disproportionality planning next year. A LEA, for example, can look 
at the varying experiences of its schools to identify which interventions worked and which did not, in order to take best practices to scale. By 
completing the cited example, LEAs can target specific professional development opportunities for schools to address areas of deficiency. The 
same holds true for the school to analyze their areas of need to request specific targeted support. 


During the 2018-2019 school year, DRTs will focus on the root causes analysis, implement concise interventions, and leverage resources for 
change. The LEA and school level team will collaborate in order to create and submit a LEA plan to the Maryland State Board of Education. 
After the plan has been reviewed and feedback given, the LEA should begin planning for the next year to address areas of need and expand on 
areas of success. 
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Appendix A. List of supportive school discipline websites and resources 


Maryland School Discipline Resource Guide: 


http://www.marylandpublicschools.org/about/Documents/DSFSS/SSSP/ResourceGuideMDSchDiscPactices011117.pdf 


PBIS Maryland: http://pbismaryland.org/ or www.pbis.org 


What Works Clearinghouse: https://ies.ed.gov/ncee/wwe/ 


Center on Response to Intervention: http://www.rti4success.org/ 


Mind-Up (the Hawn Foundation) https://mindup.org/ 


SWIFT Center: http://www.swiftschool.org/ 


Center on Great Teachers & Leaders at American Institute for Research: http://www.gtlcenter.org/technical-assistance/professional-learning- 
modules/social-and-emotional-learning-daily-life-classrooms 


Flippen Education: http://www.flippengroup.com/education/ckh.html 


Safe and Civil School: http://www.safeandcivilschool.com/services/classroom_management.php 


Character Counts Mid Shore, Inc.: www.charactercountsmidshore.com 


Johns Hopkins Bloomberg School of Public Health: http://www.jhsph.edu/research/centers-and-institutes/center-for-prevention-of-youth- 
violence/field_reports/PBIS 


USA Mental Health First Aid: https://www.mentalhealthfirstaid.org/cs/now-time-project-aware-mental-health-first-aid-grants/ 


Move This World: http://movethisworld.org/ 
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Violence Prevention Works: http://www.violencepreventionworks.org/public/index.page 


Center for Dispute Resolution University of Maryland Francis School King Carey School of Law: 
http://www.law.umaryland.edu/RestorativePractices 


Committee for Children: http://www.secondstep.org/ 


http://www.casel.org/guide/programs/second-step/ 


United States Department of Justice — Supporting Safe School: www.cops.usdoj.gov/Default.asp? Item=2687 


National Association of School Resource Officers: www.nasro.org/ 


Maryland Association of School Resource Officers: www.masro.com/ 


Why Try: http://www. whytry.org/ 


Botvin Life skills Training: http://www.lifeskillstraining.com/ 


Check & Connect Website: http://www.checkandconnect.umn.edu/ 


Implement Check and Connect with Fidelity Website: http://checkandconnect.umn.edu/manual/default.html 
Center for Dispute Resolution University of Maryland Francis School King Carey School of Law: http://www.law.umaryland.edu/PeerMediation 


Student and Youth-Based Mediation: http://www.theresolutioncenter.com/peermediation/ 
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Appendix B: Quality Improvement process resources 


Anderson, A. (2005). An introduction to theory of change, Evaluation Exchange, 11(2), 12-19. http://eric.ed.gov/?id=ED486082 


Bocala, C., Henry, S. F., Mundry, S., & Morgan, C. (2014). Practitioner data use in schools: Workshop toolkit (REL 2015-043). Washington, DC: 
U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional 
Educational Laboratory Northeast & Islands. http:// eric.ed.gow/?id=ED55 1402 


Cherasaro, T. L., Reale, M. L., Haystead, M., & Marzano, R. J. (2015). Instructional improvement cycle: A teacher’s toolkit for collecting and 
analyzing data on instructional strategies (REL 2015—080). Washington, DC: U.S. Department of Education, Institute of Education Sciences, 


National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central. 
http://eric.ed.gov/?id=ED556126 


Cokeley, S., Byrnes, M. A., Markley, G., & Keely, S. (2007). Transformation to performance excellence: Baldrige education leaders speak out. 
Milwaukee, WI: ASQ Quality Press. 


Ewy, R. W., & Gmitro, H. A. (2010). Process management in education: How to design, measure, deploy, and improve educational processes. 
Milwaukee, WI: ASQ Quality Press. 


Gerzon, N., & Guckenburg, S. (2015). Toolkit for a workshop on building a culture of data use (REL 2015-063). Washington, DC: U.S. 
Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional 
Educational Laboratory Northeast & Islands. http://eric.ed.gov/?id=ED555739 


Kekahio, W., Lawton, B., Cicchinelli, L., & Brandon, P. R. (2014). Logic models: A tool for effective program planning, collaboration, and 
monitoring (REL 2014-025). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. http://eric.ed.gov/?id=ED544779 
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Malone, N., Mark, L., & Narayan, K. (2014). Understanding program monitoring: The relationships among outcomes, indicators, measures, and 
targets (REL 2014-011). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education 
Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. http://eric.ed.gow/?id=ED544758 


Shakman, K., & Rodriguez, S. M. (2015). Logic models for program design, implementation, and evaluation: Workshop toolkit (REL 2015-057). 
Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional 
Assistance, Regional Educational Laboratory Northeast & Islands. http://eric.ed.gov/?id=ED556231 


University of Chicago, Consortium on Chicago School Research. (2014). Selecting effective indicators: College Readiness Indicator Systems 
(CRIS) resource series. Chicago, IL: Author. http://eric.ed.gow/?id=ED561863 
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Appendix C: Technical Assistance and Implementation Timeline 


Overview of the Maryland Model Form Disproportionality Teams Continue Disproportionality Teams 
Local Education Agencies received data and Root Cause Analysis Root Cause Analysis 

technical assistance 

Local Education Agencies and schools receive | Identifying Interventions Identifying Interventions 

technical assistance to address 

disproportionality 


Leveraging resources for change Leveraging resources for change 

Begin planning an outline for next year LEAs create and submit a plan to the Maryland 
State Board of Education 

Begin planning an outline for next year 
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Appendix D: How to analyze discipline data 


Step One: Removal Rates 


To measure disproportionality, it is necessary to first determine the removal rate for each student group within a school. A removal rate is 
calculated by dividing the unduplicated number of students in a particular student group suspended or expelled in a school by the number of 
students in that student group enrolled in the school. Removal data will be disaggregated within each school only for student groups with 10 or 
more students enrolled (e.g. African American, Hispanic, other racial/ethnic groups, and students with disabilities, etc.). 


Unduplicated Count of Students from the Student Group Removed 


Total Count of Students in the Student Group 
Unduplicated Count of Students from the Student Group Removed 


Totel Count of Stedents in the Student Group X 100 


Student Group Removal Rate = 
Student Group Removal Rate = 


For example: 


e 200 Hispanic students are enrolled in a school and 5 of those students were suspended or expelled at least once [(5 / 200 = 0.025) * 
100 = 2.5%]. Therefore, the removal rate for Hispanic students in that school is 2.5 percent. 


e 25 students with disabilities are enrolled in a school and 4 of those students were suspended or expelled at least once 
[(4/25 = 0.160) * 100 = 16.0%]. Therefore, the removal rate for students with disabilities is 16 percent. 


The Maryland model only includes data for student groups that meet or exceed the minimum n-size of 10 enrolled students. The MSDE selected a 
minimum n-size of 10 for two related reasons. First, it represents a proper balance between concerns about student privacy and accountability. If 
the n-size is too small, there is a risk that individual students may be identifiable in data that is available to the general public. However, if the n- 
size is too large, there is a risk that a larger number of schools will be excluded from the analysis. Second, a minimum n-size of 10 aligns with 
current data suppression practices. The MSDE uses an n-size of 10 when publishing data on its website for public reference, such as the Maryland 
Report Card. 


Additionally, the Maryland model only includes data when the number of removals in the student group meets or exceeds a minimum cell size of 
three. In contrast to n-size, which focuses on the number of students in a particular student group who are enrolled, cell size reflects the number of 
students in a particular student group who are removed from school. For example, if only one student in a student group is removed from school, 
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then it will not be incorporated into the analysis because it does not meet the minimum cell size of three. This minimum value prevents 
identification of schools that remove a small number of students and which may also have small n-sizes. 


Step Two: Risk Ratio Measure 


Once each student group’s removal rate is calculated, the MSDE will conduct disproportionality analysis using two measures to provide a 
comprehensive picture of how removals impact various student groups. One measure is the “Risk Ratio” measure, which compares the removal 
rate of each student group in a school to the removal rate of a comparison group in the school. The comparison group for the Risk Ratio is all other 
students in the school. For example, the comparison group for Hispanic students is all non-Hispanic students at the school. The comparison group 
for students with disabilities is students without disabilities. To calculate the Risk Ratio, the removal rate of a student group is divided by the 
removal rate of all other students (the comparison group). 


Student Group Removal Rate 


Risk Ratio= [71 Other Students Removal Rate 


For example: 


e If the removal rate for African American students in a school is 10 percent and the removal rate for all other students (i.e. those who 
are not African American) in the school is 5 percent then the risk of removal is two times (10 / 5 = 2.0) higher for African American 
students in that school. Therefore, the risk ratio is 2.0. 


e If the removal rate for students with disabilities in a school is 24 percent and the removal rate for all other students (i.e. students 
without disabilities) in the school is 6 percent then the risk of removal is four times (24 / 6 = 4) higher for students with disabilities in 
that school. Therefore, the risk ratio is 4.0. 


As the name implies, this measure captures whether the “risk” of removal for a student in one student group is lower or higher than the risk of 
removal for a student in the comparison group. Risk, in this context, refers to the likelihood that a student will receive an out-of-school suspension 
or expulsion. A risk ratio of 1.0 means the risk is higher and disproportionate. Therefore, the risk of an Asian student receiving an out-of-school 
suspension or expulsion is equal to the risk of a non-Asian student receiving a removal. A risk ratio below 1.0 means the risk is lower, while a risk 
ratio above 1.0 means the risk is higher, and disproportionate to some degree. 
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Step Three: State Comparison Measure 


To further develop the MSDE’s analysis of disproportionality, another measure called the “State comparison” measure will be used. It compares 
the removal rate of each student group in a school to a statewide removal rate for all students. To calculate the State comparisons, the removal rate 
of a student group is divided by the applicable statewide removal rate. The statewide removal rate is a three-year cumulative removal rate based 
on the prior three years of Maryland’s out-of-school suspension/expulsion and enrollment data. Because the frequency of removal varies between 
elementary and middle/high school, the MSDE will calculate two separate statewide removal rates, one for each of those two grade bands. 


: Student Group Removal Rate 
State Comparison Measure = —=_ J 
Statewide Removal Rate 


For example: 

e [fan elementary school’s removal rate for students of two or more races is percent and the statewide removal rate for all students in 
Maryland elementary school is 4 percent then the risk of removal for students of two or more races in that elementary school is one 
and a half times (6 / 4 = 1.5) higher than the risk of removal for elementary students statewide. Therefore, the State comparison ratio 
is 1.5. 


e Ifa middle school’s removal rate for students with disabilities is 18% and the statewide removal rate for all students in Maryland 
middle/high school is 6%, then the risk of removal for students with disabilities in that middle school is three times (18 / 6 = 3.0) 
higher than the risk of removal for middle/high school students statewide. Therefore, the State comparison ratio is 3.0. 


Based on Maryland’s out-of-school suspension/expulsion and enrollment data for school years (SY) 2013-2014, 2014-2015, and 2015-2016, the 
current statewide removal rate for elementary school is 1.53 percent and the current statewide removal rate for middle/high school is 6.75 percent. 
Importantly, the statewide removal rate for each of those grade bands will be re-calculated every three years to reflect contemporary data, allowing 
the State to monitor its collective progress over time. 


Thus, the Maryland model captures not only disproportionality among student groups within schools, but also whether a particular school is 
applying exclusionary discipline more frequently than the statewide removal rates. In this way, the Maryland model continues to build on the 
progress Maryland has achieved in decreasing exclusionary discipline overall, while adding the lens of disproportionality that is necessary to move 
the State toward equity. 
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